4.1 False Negative Sampling and Adoption of the Census Approach
In regions in which expert opinion suggests that all or many villages are probably not contaminated with mines/UXO, a sampling technique may be used to confirm that this expert opinion is in fact correct. The advantage is that only a small number of villages need to be surveyed, and if they are confirmed to be negative, the remaining villages do not need to be surveyed. As villages in the region in question are expected to be negative, the sample survey is designed to discover if any of these suspected negative villages have been falsely classified. In other words these false negative villages are contaminated, should be reported as positive villages and need to be surveyed. Hence the sampling program is known as False Negative Sampling or FNS. The Inception Mission developed the following detailed plan for implementation of False Negative Sampling.
After the field test of the questionnaire in Kampong Chhnang Province, the first Province in which the full survey procedure, including the FNS procedure, was implemented was Pursat Province. We decided to test the FNS procedure by implementing in parallel FNS methodology and for control, a full census survey in two districts. Details of the FNS survey process we used, and results of the FNS assessment, are the reported further in this section. A series of recommendations were made to slightly amend the FNS procedure to give more reliable results with a minimum of additional effort.
FNS Methodology Assessment Pursat Province
The FNS methodology is specified in
The Protocol For False Negative Sampling: Modified For The Survey In Cambodia, SOP 13. This protocol is based on the procedure described in the
Inception Mission. To compare results of this FNS test with actual village conditions, a full survey was conducted of all villages in the two districts comprising the test area.
Step 1
Discussions with officials in the Pursat Governor’s office confirmed our understanding that the areas of Pursat Province reporting the fewest problems with landmine an UXO contamination are Kandieng and Sampov Meas Districts.
Step 2
Discussions with the District chiefs of Kandieng and Sampov Meas again confirmed these two districts are relatively free of landmine and UXO contamination. However, at the district level, officials could not, or would not indicate any particular commune was probably landmine and UXO free (PF).
Step 3
To obtain a definitive opinion regarding the status of landmine and UXO contamination, it was necessary to consult with chiefs at the commune and village level.
Kandieng District (Option 2 - meet village chiefs and commune leaders)
We employed this option and met with the commune leaders and the village chiefs in their commune, at one meeting. Survey Team Leaders met with the commune leaders and village chiefs. Each village chief confirmed and signed, that to the best of his/her knowledge, the village was PF or PA or DA. When a village chief was absent (about 25% of the time), a chief from an adjacent or nearby village would sign. The commune leader also signed and stamped the form indicating his agreement. All 112 villages in Kandieng District were designated PF, and therefore the entire District was identified as the village population to be sampled. (Two of the 112 were questionable and these were both included in the villages selected for the sample survey).
Sampov Meas District (Option 1 – meet with commune leaders only)
We employed Option 1 and met with commune leaders only. Team Leaders met with the commune leaders individually. Each commune leader confirmed and signed (with official Stamp) that to the best of his knowledge individual villages were PF or PA or DA. Four villages, of 63 in the District, were designated as PA. The remaining 59 villages in Sampov Meas District were identified as the village population to be sampled for false negative classification.
Step 4
The Field Survey Manager (FSM) assigned the villages selected for the sample survey to the FNS Team. He decided to further refine the sampling rule to require, “if two or more villages are suspected positive in any Commune, then all villages in that Commune will be sampled”.
Kandieng District
For the 112 villages to be sampled, the ‘rule of 22’ requires that 21 villages be sampled. Every fifth village had to be sampled (112/21=5.3). The villages to be sampled were selected from the commune map to ensure a reasonable geographic distribution.
Sampov Meas District
For the 59 villages to be sampled, the ‘rule of 22’ requires that 19 villages be sampled. Every third village had to be sampled (59/19=3.1). A suitable map was not available as the basis for selection of villages according to geographic distribution. Instead the 59 villages were grouped according to Commune and numbered. Every third village on the list was selected for FNS.
Following this test of the FNS procedure we concluded that:
1. Sample sites for FNS should be selected based on geographic distribution.
Whenever possible as this provides a more random distribution geographically of sample sites. Often this procedure has limitations. Village centers as shown on maps are often not in the center of a village area. Village boundaries on maps are arbitrary or not known.
2. Our proposed rule is that if FNS shows two, or more, villages in a Commune to be positive, then a full survey should be conducted for all villages in the subject Commune.
3. Option 2, meeting with all village chiefs and the commune leaders in a Commune is preferred to Option 1 (meeting with the commune leader only).
Option 2 is more difficult to achieve, as it is not always possible to get all villages chiefs in a commune together at one meeting. If 75% of village chiefs are present, the meeting can be considered successful, as village chiefs from adjacent or nearby villages can be assumed to provide accurate information for those chiefs that are absent.
4. The available databases have limitations:
CMAC mapping is provided in the database on a commune basis, not a village basis. Just because a minefield is identified on a commune basis does not mean that it is applicable to every village in the commune. The CRC victims’ database provides information regarding casualties. There is some question as to whether or not the incidents reported actually took place in the village named, or if the casualties occurred elsewhere and the victims later moved to the village named. We conclude that the CRC database is a useful guide as to where landmines and UXO were present but that accurate accident locations are not a reporting priority for the CRC.
5. Expert opinion in Cambodia will have to be obtained not only at Provincial and District level, but also right down to the Commune and village chief level.
Our test shows the benefit of obtaining information at the village chief level. This detail is required in part to answer possible questions as to why we cannot rely to a greater extent on existing database information. (# 4 above).
6. FNS at best will have to be based on grouping several contiguous communes as an appropriate geographic area.
It will seldom be the case that an entire district will not have some suspect villages.
7. The results for Kandieng are considered acceptable.
(i.e. 2% of villages not sampled were positive). The results for Sampov Meas are considered unacceptable (ie 6% of villages not sampled were positive).
8. It is important to realize, and accept, that the Level One Survey provides information about a village based on villager response about the situation on the day of the survey interview.
Information from the past, although a helpful guide, may or may not be a concern of villagers currently. Also, there is no way of knowing, and the survey does not attempt to make predictions of whether villagers will find UXO in the future. (During the FSM 1% QA check we recorded an instance where a villager had found a UXO the day after the survey was conducted).
The next Provinces we surveyed were the heavily contaminated Provinces of:
- Battambang
- Pailin
- Banteay Meanchey
- Otdar Meanchey, and
- Siem Reap
In each of these Provinces the result of the expert opinion survey at the Province level was
that FNS was not applicable for that Province.
During this period the Survey was able to obtain maps from the US Government showing locations of all bombing activity conducted by US forces over Cambodia. The maps produced showed heavy concentrations of aerial bombardment, and hence a high probability of UXO, in nearly all areas of Cambodia not suspected of having dense land mine contamination.
Faced with this situation the Project team decided to apply the FNS methodology to a sample area with suspected low levels of contamination by both mines and UXO. The areas selected were in Takeo and Kandal Provinces.
Results of False Negative Sampling in Kandal and Takao Provinces
Introduction
If FNS is to be used effectively in Cambodia it would provide the greatest benefit in terms reducing the number of villages to be surveyed in areas that are relatively uncontaminated, in which the potential for reduction in the level of effort is greatest. After finding that FNS was not applicable in the north-western Provinces we needed to assess how useful it would be in the less contaminated eastern and southern regions of the country. In February 2001 we therefore conducted a pilot project with Team three in selected districts of Kandal and Takeo.
Criteria used in the selection of the Districts in Kandal and Takeo were:
- Good access
- Few suspected mine areas
- relatively few UXO
- Many villages
- Relatively large population
Methodology
The FNS methodology was followed as amended for Pursat Province. In summary:
- Selected districts were expected to be free of mines based on database information, including victim data.
- Selected districts had the least concentration of bombing based on US bombing information.
- Met with Governor’s of both Provinces to obtain permission for the survey and expert opinion regarding probability of mines and or UXO.
- The fieldwork was conducted by one Survey Team, comprised of 18 persons (2 Team leaders and 8, 2-man Detachments).
- The fieldwork required 12 workdays during the period 24 January to 06 February 2001.
- Team Leaders met with District officials to explain our activities.
- Team Leaders and Survey Detachments organized meetings in each Commune with the commune leader and the village chiefs of the Commune (Option 2)
- Each village chief indicated whether he believed his village to be probably landmine and UXO free (PF) or probably landmine and/or UXO affected (PA.).
- Approximately 75% of village chiefs attended the meetings. If a chief was absent, the chief from a neighbouring village, or the commune leader provided the opinion.
- All village chiefs present signed a statement that the information provided was the best to their knowledge and the commune leader also signed and stamped the statement for his Commune.
- Based on the information from the commune leader and village chiefs each village was designated probably landmine and/or UXO affected (PA) or probably not affected (PF).
- Communes with more than one PA village were eliminated from the FNS procedure and all villages in such Communes require a survey.
- Communes with one or no PA villages were included in the FNS village population to be sampled.
- Contiguous communes selected for the FNS sampling procedure were grouped to form as large an area as possible (i.e. as many villages as possible).
- Villages were selected for survey based on the FNS “Rule of 22”.
- In communes with one PA village, the positive village was always surveyed.
- Villages were selected on maps to ensure good geographic distribution.
- Survey Detachments surveyed the selected villages.
- If two villages in a commune were found to be positive, no further sample surveys were conducted in the Commune, instead all villages in that Commune were surveyed.
Results
The results of the false negative sampling in Kandal and Takao Provinces are summarized in
Table 1 below:
Table 1
|
|
|
|
Villages Identified
|
|
|
|
|
|
|
|
|
|
|
|
as +ve (PA) or –ve (PF)
|
PF village Population
|
|
Villages Surveyed
|
|
|
|
|
By Commune/village chiefs
|
to be Sampled
|
|
|
|
Province
|
District
|
Code
|
Positive (+) UXO
|
Positive (+) Mines
|
Negative (-)
|
Yes
|
No
|
Villages Selected for FNS
Sample Survey
|
Yes
|
No
|
Positive
|
Negative
|
Takao
|
Prey Kapbas
|
2106
|
15
|
|
95
|
30
|
80
|
18
|
18
|
|
|
|
|
Angkor Borey
|
2101
|
6
|
|
28
|
24
|
10
|
9
|
9
|
|
|
|
|
Borey Cholsar
|
2103
|
13
|
2
|
24
|
|
39
|
|
|
|
|
|
|
Koh Andeth
|
2105
|
17
|
|
51
|
|
68
|
|
|
|
|
|
Kandal
|
Sa-Ang
|
0810
|
24
|
1
|
94
|
76
|
43
|
20
|
20
|
|
|
|
|
Koh Thom
|
0804
|
15
|
1
|
77
|
68
|
25
|
28
|
28
|
|
2
|
|
|
Lerk Deck
|
0805
|
14
|
1
|
9
|
7
|
17
|
3
|
3
|
|
1
|
|
|
|
Total
|
|
104
|
5
|
378
|
205
|
282
|
78
|
78
|
|
3
|
|
Summary Statistical Results:
- There are 487 villages in these seven Districts
- 22.6% of the villages were definitely or probably positive (DA or PA), almost entirely due to Spot UXO, the remaining 77.4% of villages were probably negative (PF).
- 42.1% of the villages were included in the population of villages to be sampled based on the FNS procedure.
- Only a little over half of PF villages were included in the population of villages eligible to be sampled using the FNS procedure (205 of 378 PF villages). Many negative villages are in Communes that had two (2) or more positive villages and therefore could not be included in the population of villages eligible for the FNS sampling procedure.
- Four of every 10 (38 %) of the villages selected for the FNS sample survey had to be surveyed. This high percentage is because areas of uncontaminated villages were generally small and the Rule of 22 requires more villages to be visited where the number of probably uncontaminated villages is small compared to situations where there are a larger number of uncontaminated villages in an area.
- 282 villages of these 487 villages could not be included in the village population to be sampled using the FNS procedure. In other words these 282 villages have to be surveyed. In addition 78 villages had to be surveyed using the FNS procedure. The total number of villages surveyed was therefore 360 of the 487 total or 74% in this area suffering the least of any area in Cambodia from mine/UXO contamination.
- The FNS procedure required 12 workdays for one Team of 6 Detachments and 2 Team Leaders.
- The 487 villages in the 8 Districts are generally readily accessible, often in close proximity and the contamination is due almost entirely to Spot UXO (only 5 villages had suspected mined areas).
Conclusions
The FNS procedure has very limited application in Cambodia and should not be used in the survey of Cambodia because:
- There are no areas where expert opinion and database information will indicate there is probably no contamination.
- Any areas where the FNS procedure can be applied will be small and the number of village visits required by the Rule of 22 will be large and will not result in any significant time saving.
Summary
282 villages (57.9% of all villages) in the seven (7) districts will have to be surveyed. In addition a further 78 villages have to be surveyed from the village population eligible for the FNS sample procedure. A total of 360 villages or 74% of all villages will have to be surveyed.
It is estimated this will require 12 workdays for one Survey Team (8 Detachments and 2 Team Leaders), assuming 3 villages per day. The sample survey of villages based on the FNS procedure required an additional 12 workdays. As we became more familiar with selection of sample villages and the FNS rules of procedure this level of effort could be reduced somewhat. If a full survey were conducted for all these 487 villages, and assuming that each detachment could conduct 3 surveys per day, the full survey would require 21 days. This is 3 days less than we would have spent using the FNS procedure.
As this area represents an area of Cambodia with one of the highest expected concentrations of uncontaminated villages, and as there is no time saving even here using the FNS sampling approach, the L1S team concludes FNS will not produce any significant time saving in Cambodia, and a full census approach should be adopted for the entire country.
